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Abstract: In this paper, we consider the state-dependent reflecting random walk 
on a half-strip. We provide explicit criteria for (positive) recurrence, and an explicit 
expression for the stationary distribution. As a consequence, the light-tailed be- 
havior of the stationary distribution is proved under appropriate conditions. The 
key idea of the method employed here is the decomposition of the trajectory of the 
random walk and the main tool is the intrinsic branching structure buried in the 
random walk on a strip, which is different from the matrix-analytic method. 
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1 Introduction and Main results 

Let d > 1 he any integer and denote ^ = {1,2,- ■■ ,d}. We consider the reflecting 
space-inhomogeneous and state-dependent reflecting random walk on a half-strip S = 
{0, 1, 2, . . .} X ^. This model is often referred to as the state-dependent quasi-birth-and- 
death (QBD) process in queueing theory. Studies on the state-dependent QBD process 
has been centered at its stationary distribution such as properties of the rate matrices, 
efficient algorithms for computations, often through the matrix-analytic approach or the 
censoring techniques (e.g. [12], [2] and [16]). In this paper, we propose a different method 
to decompose the trajectory of the random walk on the strip using the intrinsic branching 
structure ( [Hj), through which, we provide criteria for (positive) recurrence, obtain an 
expression for the stationary distribution of the walk, and characterize the exponential 
tail asymptotic behavior of the stationary distribution for the walk. 
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We consider the random walk Xn = (^n, ^n), n = 0, 1, . . ., on a half-strip, which is a 
Markov chain with C,n ^ {0, 1, 2, . . .}, referred to as the layer (or level), and Yn G ^. Let 
Li = {{i,r); r = 1,2, ■ ■ ■ ,d}. Then, the half-strip S can be expressed as S* = U^Lj. 
The transition probability of the walk is given by 

/ Po 

Ri Pi 
Q3 



\ 



Qi 



P2 
R3 



P3 



\ 



(1.1) 



J 



where Pq is a d x d stochastic matric and {{Pn, Qn, Rn), n G Z"*" = {1, 2, . . .}}, satisfies 
{Pn + Qn + -Rn)l = 1 with 1 being a column vector of ones. For a matrix A, its (z, j)th 
component is denoted by A{i,j), 1 < i,j < d. Following |T], we assume the following 
conditions, under which the process is irreducible: 

Ci. log(l - \\Rn + PnWy^ < oo and log(l - + g„||)"i < oo; 
C2. For any n and any j, ^2^=1 Pn{hi) > and ^^=1 Qn{i,j) > 0; 
C3. The layer is in one communication class. 



To state the main results, we introduce the following notation or definitions: = 
( 0, . . . , 0, 1, 0, . . . , Oj ; I A is the indicator function of the set A; B' and b' are the transposes 

ith component 

of matrix B and vector b, respectively; 1 = (1, 1, ■ ■ ■ , 1)'; P^„{-) represents the probability 
given that the random walk starts from layer n with the distribution P(„ j)(-) means 
the probability given that the walk starts from the site (state) {n,i); E^^{-) and E(^n,i)i') 
are similarly defined. 

Define the hitting times T„ and T+ by T„ = inf{t > : Xf E Ln} and T+ = inf{t > 



1 : Xt G Ln}, respectively. Define fx^y and f^^y respectively as fx^y = Px{Ty 



in) 



n] 



Px{Xn eLy, Xm^Ly, 1 <m <n) and fr,^y = Xl^^i /i"?/- Also, define 



Ex{T;j 



X]n=i ^f^J if f^,y ~ 

if f^^y < 1. 



-00 



Definition 1. The layer y is (layer) recurrent if fy^y = 1, otherwise it is (layer) transient. 
If Ey{Ty) < +00, the layer y is called (layer) positive recurrent. 

Define recursively for nG{0,l,2,...}, 

Co+ = Po and C+ = (/-g„C+_^-P„)-ip„, n = l,2,.... (1.2) 



The existence of (/ — QnCn-i ~ Rn) ^ is a consequence of assumption (C2). Note that 
Po is stochastic, so is (see P). Define also Uq = 1, and for n > 1, 

A+ = (/-g„cti-i?n)"'Qn and n+ = (/-g„cti-i?nr'i. (1.3) 
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Remark 1.1 Ctihj) has an interpretation as the probability of the random walk starting 
from {n,i) and with the reflection at layer reaches layer n + 1 at point {n + 
often referred to as the exit probability and denoted as rjn as well in our model. Also, 

can be interpreted as the expected number of steps from {n,i) to (n — 
caused by a step from layer n + 1 to {n,i) (see / I2.3p . i.e., the mean offspring of the 
'father" step from layer n+1 to (n, i) ). It is worthwhile to mention that the rate matrix 
and the fundamental period matrix are key probabilistic quantities in studying the level- 
independent QBD process. They are generalized into two matrix sequences and G~, 
respectively, when the method is used to study the lev el- dependent QBD process (e.g., [W^ 
and JW^). Their dual versions R~ and also play an important role in the study using 
the matrix- analytic method (e.g. fT^). The matrix is the same as G^, while A^ is a 
unique quantity from the branching process method, which is not a usual measure used in 
the matrix- analytic method. 

The first group of results are conditions for recurrence and positive recurrence of the 
walk. 

Theorem 1.1 For the random walk starts from layer with an initial distribution fi, 
define 

+ 00 

P^ = Y,t^kAtAt_,...Atl, 

k=0 

where fik = /^Co^Ci" ' ' ' Ck-i- Then the random walk is recurrent if and only if [5^ = oo. 

Remark 1.2 Actually, (5^ in the Theorem is the expectation number of the visiting times 
by the random walk at layer , which can he calculated by the means of the intrinsic 
branching structure within the walk. 

To state the criteria for the positive recurrence, we need the "exit probability" from 
the other direction. Let Z = {0, ±1, ±2, . . .} and a G Z. For n < a, define recursively 

Ca,a = P and C-^ = {I - PnQ+i^ a- Rn)~^Qn, U < a, (1.4) 

where p is stochastic, i.e., pi = 1. Then under condition G, the limit (~ = hma_>.oo Cn a 
exist and satisfy the following equation (Theorem 1, pQ), 

(- = {I-PnC-^,-Rnr'Qn, ^ G Z. (1.5) 

Define for n > 1, 

A- = iI-P^C-^,-Rr,r'P^ and = (/ - P^C+i - i^J-^l. (1.6) 

Remark 1.3 Cni^^j) ^'^ interpretation as the probability of the random walk starting 
from {n, i) reaches layer n — \ at point {n — which is the same as G^. Also, A~{i,j) 
is the same as A^{i,j) but from the other direction, can be interpreted as the expected 
number of steps from {n,i) to {n + l,j) caused by a step from layer n — 1 to {n,i), 
which is unique to the branching process method. 
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For convenience, let Uq = 1. 



Theorem 1.2 For the random walk starting from layer with an initial distribution fi, 
define 



gt = I'Po ( 5^ A" ■ ■ ■ ^fe-iu, ) + d. 

\k>l j 

Then the random walk is positive recurrent if and only if gf < oo. 

Remark 1.4 Actually, g'^ = ijlPq (X]fc>i ^1^2 ' ' ' ) + A*-'- expectation of the 

first return time of the random walk start at layer , which can be calculated by means 
of the intrinsic branching structure within the walk. Note that g'^ < gf, the criteria is 
independent of the initial distribution of the walk start at layer 0. 

When the walk is state- independent, i.e., (P„, Qn, Rn) = (-P, R) for n > 0, we denote 
the walk as {X„, n > 0}, and have correspondingly 

C" = (/ - PC" - R)-'Q, 

and 

A- = {I - PC - Py^P, W = {I- PC - Py^l- (1.7) 

Corollary 1.3 Suppose that the random walk ra > 0} starts from layer with an 

initial distribution Jl. Then 

(1) The random walk is positive recurrent if and only if 



Q+ = VP [A^f^^ u-) + d< 00, 



k>l 



(2) Denote the maximum eigenvalues of A as \a- ■ Then \a- < 1 whenever gf < 00. 

We now state the main result for the stationary distribution. We assume that the walk 
is positive recurrent and start from layer with a "proper" distribution. The so called 
"proper" distribution is the "censored measure" , a terminology borrowed from queueing 
theory (e.g. [IB]). Define Pi by 



Pi 



/ Pi Pi \ 

Q2 R2 P2 

Q3 R3 P3 



Let 5*0 = Lq and let Si = S/Sq be a partition of the state space S. Then P can be 
partitioned according to Sq and Si as 



P 



po U 
D Pi 
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where P° = Rq, U = (Pq, 0,0,.. .) and D = {Qi, 0,0,.. .f. 

The censored matrix p('^o) of P with the censoring set 5*0 is defined by 

p{So) = po ^ UP^D, 

where P^ = ^X=q{P^Y is called the fundamental matrix of P^. P^^^) is a. d x d matric, 
and the censored matrix p('^o) has a probabilistic interpretation: it is the probability that 
the next state visited in Sq is j, given that the process starts in state i E Sq. 

A measure flo which satisfies 

tloP^^'^^ = flo (1.8) 
is called as censored measure with censoring set 5*0. 

Theorem 1.4 If Qi < oo and the walk starts from layer with the censored measure 
flo, then the stationary distribution {z/„, n = 0, 1, 2, . . .} exists and unique, which can be 
expressed explicitly as 

floPoA^A^ ■ ■ ■ A-_^u- 

n > 0; (1.9) 



/^o A(Efc>i A ^2 ■ ■ ■ \-i^k ) + /^ol 
and 

where /io^o(Zlfe>i ^1^2 ' ' ' ^k~i^k) + P-q'^ < ot < 00 and u~ = {I - P„C+i - ^n.)"^- 



Remark 1.5 We can show that the expression in Theorem I.4 is consistent with the 



matrix-product form solution given by the matrix- analytic method: 

To see it, we notice that u~ = (I — PnCn+i ~ Rn)~^ is the entry Pi^ of the fundamental 
matrix. Then, according to 

R^ = P 1 p(") 

we can have the equivalence. For details, readers may refer to fT^ . [T^ and [TB^ . 



The expression of the stationary distribution for the state-dependent walk in Theo- 
rem II .41 enable us to obtain the following asymptotic behavior. Let D = {(P, Q, R) : (P + 
g + P)l = 1, gt < oo}- 

Theorem 1.5 For the random walk on a strip, 
(1) // (P, Q, R) G D, we have Xa- < 1. 
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(2) Suppose that the random walk starts from layer with the censored measure fiQ, 
and the transition probabilities satisfy {Pn, Qn, Rn) {P, Q,R) as n ^ oo with 
{P,Q,R) G D. Then the random walk is positive recurrent and the stationary 
distribution {z/„, n > 0} defined in U.9\) is light-tailed, with the decay rate < 
A^- < 1 along the layer direction, that is, for each fixed 1 < j < d. 



lim 

n— >oo 



n 



log A, 



:i.ii) 



where \a- is the maximum eigenvalues of A (given in(l.D). 



Example 1.6 (A retrial queue with a state-dependent retrial rate) This model is 
a modification of the standard M/M/c retrial queue (for example, see Falin and Temple- 
ton [5]). In the modified model, instead of the retrial rate n6, we assume the total retrial 
rate is 6n, where n is the number of customers in the retrial orbit. For this model, let N{t) 
and C (t) be the number of retrial customers in the orbit and the number of busy servers at 
time t, respectively. Then, it is easy to see that {N{t), C{t)) is a continuous-time Markov 
chain. We show how to apply Theorem 1 1.51 to obtain the exponential decay rate. For this 
purpose, assume that 6*^ — > 6' < oo as n — )■ oo. Then, the generator of the limiting chain 
is given by 



Q 



Bo 
C 



A 
B 
C 



\ 



A 
B 



A 



:i.i2) 



where 



B 



A 

-(A + /i + 



/ 



A 



\ 



A 



and 



Without loss of generality, we assume \ + 
convert the generator to a transition matrix P ■ 



\ 



(c-l)/i -[A + (c- 1)^ + ^] 



A 



/O 



C 



e 



-(A + c//) j 
\ 



\ 



J 



- 1. Upon uniformization, we can 
Q to have {P,Q,R). To determine 



the condition for positive recurrence and A^-, we use Theorem 11.41 and (11. 7p . respectively. 
First, (I1.7P is equivalent to the equation i?+ = P + R^R + R^^Q. To solve this equation, 
we notice that 

^00- 



R-" 





ri r2 
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which greatly simphfy the calculations. Also, we can find that the chain is positive 
recurrent if and only if < 1. For example, when c = 1, Xa- = = A(A + 9)/fj,9 and 
when c = 2, 

^ ^ A {X + 9)^ + 9fi 
^~ e^i 3\ + 2fi + 29' 

As c gets larger, the formula becomes cumbersome and is less interesting. 

We arrange the remainder of this paper as follows. As the main tool of this paper, 
the intrinsic branching structure within random walk on a strip is briefiy reviewed in 
Section [21 and then the proofs for the Theorems are followed in Section [31 

2 A brief review for the intrinsic branching structure 
within random walk on a strip 

The intrinsic branching structure within a random walk is a very powerful tool in the 
research on the limit property about random walk. For the neighborhood nearest random 
walk on the line, Dwass ([1], 1975) and Kesten et al. (jlO], 1975) observed a Galton- 
Watson process with a geometric offspring distribution hidden in it. Kesten et al. (|10j) 
proved a stable law by using the branching structure for the random walk in a random 
environment. For other random walks, e.g., the random walk with bounded jumps, the 
branching structure were revealed by Hong Sz Wang ([B], 2009) for the (L, l)-case and 
Hong & Zhang ([7], 2010) for the (l,i?)-case. 

The intrinsic branching structure within random walk on a strip has been revealed by 
Hong & Zhang ([8], 2012), which enables us to provide explicit criteria for (positive) 
recurrence and to obtain an explicit expression for the stationary distribution. As a 
consequence, it allows us to consider the tail asymptotic of the stationary distribution. 
The key point is the trajectory decomposition for the random walk. If the walk starts 
from layer n > 0, the trajectory has the "upper" and "lower" parts, which are introduced 
in the following subsections. 

2.1 The "lower" branching structure 

Assume that Xq G L^, the random walk starts from layer k with initial distribution /i^ 
or /ife(z) = -P(^o = k, Yq = i). For < n < k and i G {1,2, ■■ ■ ,d}, define as the 
number of steps from layer n to {n — before the hitting time T^+i, and as the 
number of steps from layer n to {n,i) before Tk+i- Define 

Un = {u^, u^, ■ ■ ■ , u^), z„ = {z^, zl, ■ ■ ■ , Z^), 

and |U„| = U„l, |Z„| = Z„l. 

Theorem 2.1 (Hong & Zhang, 2012) Suppose that Condition C is satisfied, and the 
random walk starts from layer k with initial distribution fi^. Then {|U„|, 1 < n < k} 
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and {|Z„|, 1 < n < k} are inhomogeneous branching processes with immigration. The 
offspring distribution (1 < n < k) is given as: 

P(|U„|=m|U„+i = e,) = e,[(/-i?„)-ig„C-i]"^(J-i?„)~ip„l, 

P(|Z„| = i^|U„+i = e,) = e,[iI-Q^^C^,r'R,,f{I-Qr,C.,r'Pn-i-, 

with immigration 

P(Ufc+i = e^) = /ifc(z), t e 

□ 



The key idea in the construction of the branching mechanism is that the position of 
the walk corresponds to the time of the branching process. |U„| as the number of steps 
from layer n to n — 1 layer is indeed the {k — n)-th generation of the branching process. 
The condition T^+i < oo is obviously satisfied in our reflecting model. 

Proposition 2.2 Denote A^* as the number of steps visited at {n,i) before time Tk+i, 
and Nn = {N^,N^, ■ ■ ■ ,N^) with |N„| = N„,l. Suppose that Condition C is satisfied, 
and the random walk starts from layer k with initial distribution /ifc. Then for any < 
n <k, 

E^,(N„) = ^^,AlAt_,■■■Al^,Al^,{I-Q^CU-R^)-\ 



E,,mn\) = /XfcA+A+_r--^n+2^n+lU+, (2.1) 

and for n = 0, 

E,,{No) = fikAtAt, ...At, E,,i\No\) = f^kAtAt, ■ ■ ■ Afl. (2.2) 
Proof. We provide key steps here and readers may refer to [8] for details. Note that 



+ 00 



E(U„|U„+i) = - ^nr'QnCtir~\l - Rn)-'Qn = lJn+lA^, (2.3) 



m=l 

+00 



E(z„|u„+i) = u„+i^[(/-g„c„i)-X]'"-i(/-g„C„i)-ip„ 



A'=l 



With the help of the branching structure in Theorem 12.11 we have for any < n < k, 

E^,(N„) = E,,(U„C_i + Z„ + U„+i) (2.4) 

= -^Mfc [-^A«fc(Un|Un+i)C„i + E^^{Zn\\Jn+l) + -E^^ (U„+i | U„+i)] 
= E^^(lJn+l){I - <5nCn-l - -Rn)~\ 

and then E^^(|N„|) = P^^jU„,+i)u„. 

For 77, = 0, the expected number of steps visiting the reflecting layer before time T^+i 
is ^^,(No) = ^^,(Zo + Ui) = ^^,(Ui), and then S^,(|No|) = ^^,(Ui). Together with 

E,,(U„+i) = E^jE^,(U„+i|U„+2)] = i?,,(U„+2)A++i = ■ ■ ■ = fikAtAt_, . ..At^,A^+„ 
the proof is finished. We usually denote u,^ = 1 to make the expression uniform. □ 
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2.2 The "upper" branching structure 

Assume that Xq G L^, the random walk starts from layer k with initial distribution /i^ 
or /ifc(z) = P{C,o = k, Yq = i). Similarly, For n > k + 1 and z G {1, 2, ■ ■ ■ , d}, define 

W^ = {WlW^,--- ,W:^), and = Z-^, ■ ■ ■ , Z^^J, 

where is the number of steps from layer n to {n + 1, i) before the hitting time T^-i', 
and is the number of steps from layer n to {n,i) before the hitting time T^-i- 

Define |W„| = W„l and |Z-| = Z"!. 

Theorem 2.3 (Hong & Zhang, 2012) Suppose that Condition C is satisfied, and 
the random walk starts from layer k with initial distribution fi^ and Tfc_i < +oo. Then 
{|W„|, n > k + 1} and {|Z~|, n > k + 1} are inhomogeneous branching processes with 
immigration. The offspring distribution (n > k + \) is given by 

P(|W„| = m| W„_i = e,) = e,[(/ - i?„)^'^nC+i]'"(/ - ^n)-'Qnl, 

P{\Z-\ = K\Wn-l = e-) = e,[(/ - P^Q+ir' Rnf {I - PnC+l)~'Qnl, 

with immigration 

P(Ufc_i = e,) =/ife(z), te^. 

□ 



In parallel, for n> k + 1, denote 

iV-'* = tt{^e [0,T_i) : X, = (n,z)}, 

where N"'"^ is the number of steps visited at {n,i) before the hitting time Tk-i- Define 
N- = {N-'\ N-^\ ■ • • , iV-''^) and |N-| = N"!. 

Proposition 2.4 Suppose that Condition C is satisfied, and the random walk starts 
from layer k with initial distribution fi^. Then for any n > k + 1, 

E,^{N~) = E^,(W„_0(/-P„C+i-^n)"' 
E,^{\N-\) = E,^{Wn^,){I-Pr,Q^,-Rn)-H = f,,A^A^^,---A-_,A~_,u~. 

(2.5) 
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3 Proofs 



3.1 Criteria for recurrence — Proof of Theorem LLIJ 

Let T° = 0, and let = inf{n > T^-^ : X„ e Ly} for A; > 1, or is the time of 
the k-th return to layer y. Note that Ty > 0. Hence, a possible visit at time does not 
count, and Tj equals to defined above. 

We firstly extend a basic property about Markov chains to the random walk on a 
strip, which is stated in the following lemma. 

Lemma 3.1 Layer y is recurrent if and only if Ey{\lSiy\) = +oo. 

Proof. Denote fx^y = PxiTy < oo). Then, 

PxiT^ < = fxj'y-'. 

This is clear, since in order to visit layer y for exactly the fc-th time, the walk has to go 
from layer x to layer y first, and then return to layer y k — 1 times. A detailed formal 
proof is similar to that for Theorem 3.1 in [3] for the random walk on a line. 

Recall that |Nj^| = Nj,l = X]m=i h^m^Ly} is the number of visits to layer y at positive 
times. By the definition, layer y is transient if and only if fy y < 1. Suppose that layer 
y is transient, then 

+ 00 +00 

^.(|N,|) = 5^P..(N,l>A;) = ^P,.(r^^<oo) 

k=l k=l 

+ 00 J, 

ixjy^y = ,r < +00- 

k=i ^ ■Jy'y 

Thus, layer y is recurrent if and only if i?y(|Ny|) = +oo. □ 

Proof of Theorem \1.1\ Because the random walk is irreducible, we only need to 
calculate the i?^(|No|), where the walk starts at layer with distribution /i, and |No| = 
l(^iGLo) is the occupation time of the walk at layer 0. We can decompose the 
trajectory of the walk as the summation of infinite "pieces" , each "piece" is an immigration 
("lower") branching structure as considered in Theorem 12. 1[ In fact, by recalling the 
definition of the hitting times = infji : Xi G L^} for layer k and denoting X^'^''^ = 
{Xi,Tk <i < Tfc+i}, we can write 

+ 00 +00 

{X,,t>0}= [j{X,,Tk<t<Tk+i} = [jx'^^'^\ (3.1) 

k=0 k=0 

and as a consequence, 

+00 +00 7fc-|-i +00 

|No| = J2 kx,eLo) = E E l(^-^^o) = E |Nol^^'=\ (3.2) 

i=0 k=0 i=Tk k=0 
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where the superscript is used to emphasize that the process starts at T^. For k = 0,1, 
each trajectory "piece" X^'^''^ = {Xi,Tk < i < Tfc+i} formulates a branching 
structure with immigration P(\Jk+i = e.^) = //^(i), where fj,k = /^CcTCi' ' ' 'Ck-i- By fl2.2p 
of Proposition 12.21 we have, 

Combining with ( I3.2p . 

+00 +00 

k=0 k=0 

The proof is complete. □ 



3.2 Criteria for positive recurrence — Proof of Theorem \LM 

Define as the return time of layer n when the random walk starting from layer n, 

T„" the hitting time of layer n when the random walk starting from (n — 1, j), and 

T„ ' the hitting time of layer n when the random walk starting from (n + 1, j). Then 
by the path decomposition, 

j j j 

and therefore, 

= Ef^nC^ ^Xi = {n~l,j){Tl '^^ + 1) + ^ /xi=(n+lj)(7'i '^^ + 1) + ^ Ix-, = (n,j)) 
j j j 

= 5^P^„(Xi = (n - 1, j))(i?(n-i,,)T+ + 1) 
i 

+ 5^P^„(Xi = (n + l,j))(i?(n4-i,,)T+ + 1) + 5^P^„(Xi = (n,j)). 
i j 

Note that 

n—l +00 

E(„_i,,)(7^+) = ^i?(„_i,,-)(|N,|) and E(„+i,,)(r+) = ^ E(„+i,,-)(|N-|). 

fc=0 fc=n+l 

It follows from (^J^) and ([23]) that 

n-l 

fc=0 

+00 

fc=n+l 
11 



Hence we have 

n-l 



= /x„g„(5^A+_,A+_2---A+^iU+) + 5^P,„(Xi = (n-l,j)) 

k=0 j 

+00 

+/i„P„( ^n+l<+2---^fe-l%) + 5^P^„(Xi = (n+l,j)) 

fc=n+l j 

+ ^P^„(Xi = (n,j)) 

n— 1 +00 
fc=0 fc=n+l 

Particularly, if the random walk starts from layer with an initial distribution /i, we 
have 

E,{T^) = /iPo(5^ A^A^ ■ ■ ■ A^_,n^) + /.l. (3.3) 

k>l 

Thus the reflecting random walk on a strip is positive recurrent (independent of the initial 
distribution /i) if and only if f?!" < 00 . □ 



3.3 Stationary distribution — Proof of Theorem\1.4 



Suppose that the random walk starts from layer with a censored measure p,Q, which 
satisfies 

/ioP(^«^ = iio. 

The following lemma modifies Thm 5.4.3 in [3] about a stationary measure of a general 
Markov chain on Z'^ to our model, and defines a stationary measure for the random walk 
on a strip. 

Lemma 3.2 Suppose that the random walk starts from layer with a censored measure 
/io, and layer is a recurrent layer. Then {Un, n G N} defines a stationary measure, 
where 

/Tq-^ \ +00 

Etio E hxr^=M} = E ^Mo(^m = {n, z), m < T+). (3.4) 

\ m=0 / m=0 



Proof. The key idea of the proof is to use the "cycle trick" . Vn{i) is the expected number 
of visits to {n,i) at times 0, 1, . . . ,Tq — 1. And Xlj; (^'O] is the expected 

number of visits to (n, i) at times 1,2,..., Tq, which equals to Vn,{i) since Xrp+ ~ fiQ if 
Xq ~ /io based on the property of p.Qp^^°^ = 

The goal is to prove that z7„ defined in (13.41) is a stationary measure, that is, 

Y^y{3)p[{yd).{n,i)\ =Vn{i). (3.5) 
yj 
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By Fibini's theorem, we get 

+00 

^'i^yU)p[{y,j),in,^)] =^^Ptioi^m = {y,j), m<T+)p[{y,j),{n,i)]. 
Case 1: n ^ 0. In this case, we have 

y,j 

= ^Pfioi^ni = iy,j), m < To+, = (n,i)) 

yJ 

= PfioiTo > m + 1, Xm+i = {n, i)), 

and then 

+00 

^'i^yU)p[{y,j),{n,i)] = ^^Pfi,{Xm = {y,j), m <T^I~)p[{y,j),{n,i)] 
yd "1=0 y,j 

+00 

= X] (^0^ > m + 1, = (n, i)) . 

m=0 
+00 

= XI ^Hoi^m = in,i), m < To+) = z7„(z), (3.6) 

m=0 

because P^q(Tq'" > 0, Xq = {n,i)) = 0. 

Case 2: n = 0. At first, note that the process starts from layer with the initial 
distribution p,Q, i.e., the right hand side of (13. 5p is p,o. For the left hand side of (13.51) . 
we calculate 

^PfioiXm = {yd), m < T(l)p[{yJ),{n,i)] 
y,j 

= ^Pnoi^m = (yJ), m < T(l, Xm+i = (n,i)) 
yJ 

= P^,(To+ = m + l, X^+^ = iO,^)), 

and then 

+00 

^'i'yU)p[{y,j),in,i)] = ^^Pfi^{Xm = {y,j), m <T^)p[{y,j),{n,i)] 

y,j "1=0 y,j 

+00 

= ^P^„(To+ = m + l, X„+i = (0,2)). 

m=0 

Note that Tq > 1. Therefore, P^q(Tq^ = 0, Xq = (0,i)) = 0, and we have 

+00 +00 

^ Pf,, {T+ = m + l, X^+i = (0, z)) = ^ Pf,, {T+ = m, X„, = (0, i)) = flo, 

m=0 m=0 
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because /xq is the censored measure. The proof is complete now. □ 
Proof of Theorem\1.4\ First, we can calculate the stationary measure in (13. 4p by using 



the branching structure. The stationary measure is given by = Ep^^^ {Y^=q ^Xm=(n,i)} 

which is the expected number of visits to (ra, i) before time Tq (but not contains the time 
Tq). So, Vn{i) {n > 0) equals to EiN~ obtained by the branching structure in (12. 5p . 
The stationary measure {iPn, n can be expressed as 



p,o n = 0. 



Note that 

+00 

^'i^nii) = PfioiTo >m) = Ep^^T^ 

n,i m=0 

The condition gf < oo ensures that 



n,i \k>l J 

As a consequence the stationary distribution equals 



fiol = g^ < gi < 00. 



where «" = (/- P„C7+i - and 



/^o^o(Efc>i A A ■ ■ ■ A-iUfc ) + /ioi 

□ 



3.4 Light-tailed behavior — Proof of Theorem \1.5 



It is well-known that the stationary distribution for the state- independent random walk (or 
a QBD process) on a half-strip is matrix-geometric. Therefore, the tail has a geometric (or 
exponential) decay. For the state-dependent random walk on a half-strip, the stationary 
tail does not always have an exponential decay. In this paper, we provide a criterion for 
this case, which is proved here. 



3.4.1 Preliminaries 

Let B = (bij) > (which is called a positive matrix) if all 6jj > 0; and P > if 
all bij > 0. The spectrum of n x n matrix B is denoted as cr(P) = {Ai, A2, ■ ■ ■ A„}, 
where cr{B) is the set of all eigenvalues Aj G C. Define the spectral radius of B as 
p{B) = max{|Ai| : Aj G cr(P), 1 < i < n}. 
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Proposition 3.3 ( Perron's Theorem in J^) If B > is an n x n matrix, then 

(1) piB) > 0; 

(2) p{B) is an eigenvalue of B, and it is the unique eigenvalue of maximum modulus; 

(3) p{B) is algebraically (and hence geometrically) simple; 
(4) 



/ B 

lim I — — I = L > 0. 



Define || ■ || as the maximum column sum matrix norm, i.e. = maxi<j<„ X]j=i l^^jl 
for B = 

Proposition 3.4 (Krause (fllf, 94), Ostrowski 73)) Denote a{A) = {Ai, A2, ■ ■ ■ A„} 

where Aj are eigenvalues of A, and a{B) = {/ii,/i2, ■ ■ - fin}, where fii are eigenvalues of 
B. Define d{a{A),a{B)) as the optimal matching distance between the spectrums (j{A) 
and <j{B), that is, 

d{a{A), cr(i?)) = min max |Aj — /iej, 

0£Sn l<i<n 

where Sn is denoted as the group of all permutations on sets {1, 2, ■ ■ ■ ,n}. Then, for any 
two matrices A,Be M"^", we have 

d{a{A),a{B)) < A{2Ky-^\\A - B\\^ , 

where K = max{||y4||, 

Simply speaking, Proposition 13.41 tells us that there exists a permutation 9 G Sn, such 
that the maximum distance between the corresponding eigenvalues is small enough. 

3.4.2 Spectral radius 

Consider the state-independent random walk {X„, n > 0} with transition probability 
block (P, Q, R), starting from layer with an initial distribution p,. Let be the unique 
sequence of stochastic matrices satisfying 

(- = (/- PC" - Rr'Q, 

and 

A- = {I- PC - R)'^P, u- = {I- PC - R)'^!- 

Denote the spectral radius of A~ as p{A~), and the maximum eigenvalues of A~ as 
A^-- Assume that the random walk is positive recurrent {qi < 00). 
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Proof of (2) in Corollary \1.3\ By Perron's Theorem in Proposition 13. 3[ we have 
p(A~) = Xa-- The condition says gf = I'P {J2k>i + d < oo, i.e., 



-^'P-[J2^^A-)'-'(^)'~' 1 - U <00. 



•A. 



(3.7) 



On the other hand, from (4) of Proposition 13. 3[ we know 

hm i^)^-^ = L>0 

which, together with (13.71) . leads to A^- < 1. 
Let 



□ 



E 



( 1 1 
1 1 



1 1 



1\ 

1 



dxd 



Denote the maximum eigenvalues of {A — eE) as A^ , and the maximum eigenvalues 
of {A~ + eE) as A^. We then have 

p{A~ - eE) = Xj and p{A~ + eE) = Xf. 
Lemma 3.5 Suppose that X^- < 1, and for A^ = {a^j), 

e < min{mina"., -^(1 - A^-)'^}. 



^,3 



Let C = 4:{2\\A- + E\\)^-U^ . Then, 

Xa- - Ce^ < Aj < 1, and Xf < Xa- + Ce^ < 1. 



(3.9) 



Proof. For such e > 0, both A~ — eE and A~ + eE are positive matrices, and the 
definitions of A~ and A^ are meaningful. By Proposition 13.31 A~ and A^ are real- valued. 
By Proposition 13. 4[ it is not hard to get that 

d{a{A-),a{A- - eE)) < A{2K)^-U^e^ < Ce^, 

and 

d{a{A-), a{A- + eE)) < 4:{2K)^-^dh^ < Ce^, 

where C = A{2K)^-idi, K = \\A- + E\\. 

Note that < A^ - eE < A~ < A-^ + eE, then 

1> Xa- = piA~) > p(A~ - eE) = A; and A^- = p{A~) < p{A~ + eE) = A+, 
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— colorredand A^- + Cea < 1 for such e > 0. 

It is obvious that A^- — Ce^ < A~ < 1. Otherwise if A~ < \a- — Ce^, then for A^-, 
there exists no permutation such that d{a{A~), a{A~ — eE)) < Ced, due to the fact that 
A~ is the largest eigenvalue of A~ — eE. 

Similarly, A+ < \a~ + Ce^ < 1 holds. Otherwise if A+ > \a- + Ce^, then for A^"t, 
there exists no permutation such that (i(cr(A~), (t(A~ — eE)) < Ce^ holds, due to the 
fact that A^- is the largest eigenvalue of A~ . □ 



3.4.3 Light-tailed behavior — proof of Theorem 11.51 

Recall D = {{P,Q,R) : (P + Q + R)l = 1, < oo}. 

Proof of Theorem \1.5\ Part (1) of the theorem has been proved in (2) of Corollary II. 3[ 
Now, we focus on part (2). The random walk {Xn, n G Z} starts from layer with an 
censored measure JIq, and the transition probabilities: (P„, Qn, Rn) {P^ Q,R) & D as 
n — )■ oo. It is easy to find that the random walk n G Z} is positive recurrent. To 

this end, recall that A;; = {I - PnQ+i - Rn)~^Pn and A~ = {I - PC - R)'^P- Then, 
A~ — A~ as — !■ +oo because that (P„, Qn, Rn) — ^ {P, Q, R) as n ^ oo; and < oo 
follows from gf < oo as (P, Q, R) G D. 

Also we have A^- < 1 as {P,Q,R) G D. For each e > defined in Lemma [3.5[ there 
exists N, such that when n > N, 

< A- - eE < A- < A- + eE, 

and then 

(A- - eE)' < ■ ■ ■ A~N+k < (^^ + ^E)'. 

Let 

^ni^)=^^PoA^A^■■■A-_,u-i^). (3.10) 
Now we consider the first inequality. Notice 

for the given A^, therefore we have 



u 



N+k\ 



,A- -eE^ 



where Di{N) = PqA^ A^ ■ ■ ■ and = (/ - PN+kCN+k+i - ^N+k) ^- Hence 
log$^+,(z) ^ A;logA7 log/iDi(iV)(^--^^'= 



> —^"e I - Ar ^ -N+kK , 

N+k - N+k N+k ' ^ ' ' 
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By Proposition 13. 3[ there exists a positive matrix , such that 

hm {^^^r = wr. 

Together with hm u~^,^{i) = u~{i) = (/ — PC~ — R)~^{i), as a consequence, 
log/iDi(iV)(^^^^)^ w]v+fe(0 is bounded in k. Thus from fl3lT|) . 

hm inf ^^^^"'"f ^ > log XT ■ 
k^oo N + k - ^ ^ 

Note that from Lemma 1331 \a- — Ce^ < < 1, so 

hminf ^"\^7^(^^ >log(A,--C.^). (3.12) 
Similarly, for the second inequality, notice 

k ~ k 

' ' ' ^N+k — (-^e ) ( 'J^ ) ' 

therefore we have 

log$^+fc(i) _^ 
hmsup — — — — < log A J. 

fc^oo N + k 

Note that from Lemma [3.51 A^ < A^- + Ce^ < 1, so 

hmsup — — — - — < log(Ayi- + Cet). (3.13) 
k^oo N + k 

Combine (I3.12p and f l3.13p to have 

log(AA- -Ce-^)< liminf ^^l^M < Umsup l^i^^ < log(A^- + Ce^). 



Let £ — !■ 0, we get 



r log ^k{^ , , 
hm ; = log Xa- 

k—^oo K 



If {Pn,Qn, Rn) — ^ {.P,Q,R)y {P^Q^R) ^ -D, the Stationary distribution {z/„, n > 0} is 
given by 

1/ = — Ti ^ U 

/ioi^o(Efc>i ^r^^ ■ ■ ■ ^fc-iUfc ) + /ioi ' 

where the denominator /io-fo(X]fc>i ^1^2 ' ' ' ^k-i^k) + /^ol < Qi < +00. 
Thus for the stationary distribution n > 0}, we have 

logz/„(i) log<l>fc(z) 
hm = hm ; = log A^- , 

n— >oo n fc— s>oo /c 

i.e., the stationary distribution is light-tailed, with the decay rate < A^^- < 1 along the 
layer direction. □ 
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